For Engineering and Science
Representation and Misrepresentation: Tufte and the Morton Thiokol Engineers on the Challenger
Parent Collection
Description
This lecture discusses and evaluates the charge made by Edward Tufte, in his major work on visual representations, Visual Explanations, that Morton-Thiokol engineers were at fault for not using more convincing graphical representations of the risk in arguing against the launch of the ill-fated Challenger Space Shuttle.
Body
It is prima facie unethical to hold people morally responsible for what they did not do or could not reasonably be expected to prevent. So, in judging ethically a person's particular past act or omission, this condition requires knowing:
- whether the person was competent and if so, if it is relevant, to what degree
- whether the person acted voluntarily and if not, what precluded or diminished the capacity to act voluntarily; and
- what the person knew or believed, or should and could have known or believed, about the issue a at hand.
Each of these queries raises often subtle conceptual issues, the concepts involved being anything but clear, and even if we had conceptual clarity, each requires the gathering of evidence that is difficult to obtain and a parsing of it that can readily go wrong through our own biases or misconceptions.
But whatever the difficulties each presents, the set forms a triad for determining fault. Someone who knows everything about a problem at hand, acts voluntarily, and yet does wrong is judged incompetent. Someone who does wrong despite being competent and knowing everything about the problem at hand is presumed to have at least a diminished capacity to act voluntarily. Someone who does wrong despite being competent and acting voluntarily is presumed ignorant. Presuming that any two conditions are satisfied when a mistake has occurred forces us to look to the remaining condition as the source of the problem.
But if we make a judgment of fault, we need to be sure of our facts. It is wrong just to presume. Edward Tufte provides a telling example of this sort of ethical failure in his judgment in Visual Explanations aboutthe engineers at Morton Thiokol the night before the Challenger disaster.
The heart of Tufte's book, as one reviewer, Ray Duncan, puts it, is a chapter entitled Visual and Statistical Thinking...based on analyses of the London cholera epidemic of 1854 and the Challenger disaster of 1986 (Duncan, 1). Tufte gives the former as a good example of the representation of causal reasoning, the latter as a bad example. As H. Allison puts it, in a review,
Tufte's close analysis demonstrates that the engineers had the information they needed--that O-ring failure rates rose as temperature declined--but didn't display it clearly. Seven astronauts' lives could have been saved with a simple graph of previous O-ring damage level against temperature (Allison, 2) .
The necessity of perspicuous representation is seen most clearly in such cases as the Challenger, Tufte argues. The engineers at Morton Thiokol failed to display the data clearly, he claims, and so the astronauts died.
Tufte's point is that the engineers' failure led to the death of the astronauts. Had the engineers presented their data clearly, he claims, Challenger would not have been launched. We shall come to see that Tufte's analysis goes wrong in three crucial ways.
- First, he fails to satisfy (c) above, not determining what the engineers knew or believed, or should and could have known or believed, about the issue at hand. He thus supposes that they knew the temperatures at launch of all the shuttles and, assuming they acted voluntarily, infers they were incompetent. But they did not know the temperatures even though they did try to obtain that information. Tufte has not gotten the facts right even though the information was available to him had he looked for it.
- Second, he thus misidentifies the effect the engineers were concerned to prevent and so misunderstands thoroughly the argument and evidence the engineers gave.
- Third, he provides a simple graph, a scatterplot, that he thinks would have saved the astronauts' lives had the engineers presented it. But the scatterplot is fatally flawed by Tufte's own criteria. The vertical axis tracks the wrong effect, and the horizontal axis cites temperatures not available to the engineers and, in addition, mixes O-ring temperatures and ambient air temperature as though the two were the same.
But we cannot understand Tufte's mistakes and how the engineers did reason until we understand the full power and extent of Tufte's grave charge. For that we need to appreciate Tufte's thesis that essential to understanding data is its perspicuous representation. We shall then be in a position to see how Tufte misrepresents the engineers' position and thus the reasonableness--and the morality--of their recommendation.
A Brief Background
The booster rockets used to launch the shuttles were designed and manufactured at Morton Thiokol and consist of segments which stack on each other. We can picture the problem these stacking segments produced by supposing that we want to create a tall coffee cup made of plastic cups designed with indented narrow bottoms so that they fit into each other in a tidy stack. If we imagine cutting the bottom out of three cups, say, and stacking them on a whole cup, we would have a smooth outer cylinder, but coffee poured into the cup would instantly come out the sides. We can try to prevent leakage if we seal the cups where they nestle into each other with, for example, snugly fitting flexible rings, but each time we pour coffee or lift the cup, the joints would be under pressure and prone to leak. In a similar way, each segment of the rocket was seated on the one beneath it and the joint sealed with two flexible and snugly fitting O-rings made from Viton, a rubber-like material. The O-ring closest to the rocket fuel is primary and the other is secondary, for back-up.
The booster rockets create enormous pressure--1004 psi--and the O-rings must seal to prevent the fuels hot gases from blowing by the O-rings and so compromising the integrity of a booster segment, putting the flight at risk. In the launch of STS 15 (STS 51-C)in January 1985, the primary O-ring on two of the joints had been compromised by fuel blowing by and eroding them (Vaughan, 155). Only the secondary O-ring was left, holding off disaster, and though it was not eroded, blow-by had reached it. The flight was preceded by a 100-year cold, weather we could expect in Florida only once every 100 years, and although the temperature at launch was 66 °F, Roger Boisjoly, an engineer at Morton Thiokol, suspected that cold temperature might have affected the Viton, making the rings less flexible and thus less likely to seal or seal quickly enough to prevent blow-by. Calculations showed that the Viton had only warmed up to 53°F at launch.
The night before the Challenger launch the following January was to be extremely cold, perhaps as low as 18 ° Flame thrower 100-year cold--with temperature at ignition in the range of 26° F to 29 °F. In a teleconference the evening before the launch, the Morton Thiokol engineers recommended that shuttles not be flown below 53 °F, the coldest known temperature to date of the O-rings during launch--in a flight in which the O-rings came the closest to complete failure and disaster.
What happened subsequently that evening is the subject of much dispute, but any narrative will contain at least the following:
The Morton Thiokol management accepted the recommendation of their engineers not to launch Challenger and sent that recommendation onto NASA.
NASA asked for a reconsideration of the recommendation.
The burden of proof seemed to shift. Morton Thiokol was to prove that the Challenger was not flight-ready apparently under the presumption that the flight would succeed otherwise.
The managers at Morton Thiokol caucused among themselves and approved the flight--despite their engineers' recommendation and sometimes vehement opposition.
Tufte's Representation
In the very making of the recommendation not to fly below 53 °F, the engineers tied together temperature and blow-by and also, as Tufte puts it, a temperature trend. (Tufte, 49 ) O-ring failure rates rose as temperature declined (Allison). But, Tufte goes onto argue, the engineers failed to relate temperature with the compromising of the O-rings in any of the 13 charts prepared for making the decision to launch (Tufte, 45). There is thus, Tufte argues, a scandalous discrepancy between the intellectual tasks at hand and the images created to serve those tasks. As analytical graphics, the displays failed to reveal a risk that was in fact present. As presentation graphics, the displays failed to persuade government officials that a cold-weather launch might be dangerous. In designing those displays, the chart makers didn't quite know what they were doing, and they were doing a lot of it. (Tufte, 45)
Whatever the difficulties in organizational structure, group think, or technical decision-making in the face of political pressure...there was a clear and approximate cause: an inability to assess the link between cool temperature and O-ring damage on earlier flights (Tufte, 39, 40).
This inability is represented nicely, Tufte is saying, in those 13 charts. Had the engineers been thinking clearly, and known how to represent graphically what they were thinking in a clear way, they would have provided a single chart, a scatterplot that ordered the data, presenting all the flights, including those in which there was no damage, in order by temperature, the possible cause (Tufte, 49).When arguing causally, variations in the cause must be explicitly and measurably linked to variations in the effect (Tufte, 52).When we do that for variations in temperature and compromise to the O-rings, we obtain a scatterplot like this (Tufte, 45):A purist might argue that any extrapolation from the available data is undetermined, but with such an ascending curve of compromise to the O-rings as the temperature decreases from 65°F to 53°F, it would be difficult for an objective observer to deny that a flight in the 26-29 °F range would be decidedly risky. In other words, the right presentation of the relevant data, Tufte is arguing, would have revealed the risk in a way that was undeniable and so persuaded NASA not to launch.
One finds astonishment in reviews of Tufte's work. How could the engineers have been so confused as to make a recommendation that related temperature to a compromise to the O-rings, but not present data to show the relation? This astonishment is natural given Tufte's analysis of what transpired the evening before the Challenger launch. By his analysis, the engineers' reasoning was intellectually flawed--The engineers were guilty of an overriding intellectual failure (Tufte, 52). They had the correct theory and they were thinking causally (Tufte, 44), but they failed to relate variations in cause with variations in effect despite claiming such a relationship
Their presentation was representationally scandalous. The discrepancy between the intellectual tasks at hand and the images created to serve those tasks was scandalous (Tufte, 45). Though thinking causally, they 'were not displaying causally (Tufte, 44).
And their behavior was thus, unethical; though there were substantial pressures to get [the Challenger] off the ground as quickly as possible...these pressures would not have prevailed over credible evidence against the launch....Had the correct scatterplot or data table been constructed, no one would have dared to risk the Challenger in such cold weather (Tufte, 52).The engineers' failure to represent clearly the data was responsible for the Challenger disaster and thus for the death of the seven astronauts.
These are indeed grave charges, and all need examination. But we must begin with the charge that the engineers were guilty of an overriding intellectual failure. The scatterplot Tufte provides properly relates cause and effect, covering both those cases with damage and those with none. Since the engineers would have presented such a chart had they been thinking as clearly as Tufte, his argument goes, we need to ask why Tufte thinks the engineers did not present such a scatterplot. What mistakes in reasoning does he think they made that led them to represent their data so poorly--and thus cause, in some measure at least, the death of the astronauts?
Tufte's Take on the Engineers' Reasoning
Tufte's work on representation is marked by a deep insight. As he puts it, "Clear and precise seeing becomes as one with clear and precise thinking" (Tufte, 53). Putting the point negatively makes it easier to understand his criticism of the engineers. Poor representation mirrors poor reasoning and encourages and sustains it. Once we go astray in our reasoning, our visual representation not only confirms the bad reasoning which it embodies, but compounds our problems by leading us into further errors.
We can see how the charts the engineers used the night of the teleconference both displayed poor reasoning and furthered it, Tufte argues, by looking at what they do and fail to do.
First, most failed to relate cause and effect or even mention temperature and compromise to the O-rings. The very first chart goes directly to the immediate threat to the shuttle and displays information about the various kinds and degrees of compromise to the O-rings, but has nothing about the claimed cause, temperature (Tufte, 40). The next chart shows how 'erosion in the primary O-ring interacts with its back-up, the secondary O-ring,' but, again, the effect is not linked to the claimed cause, temperature (Tufte, 41). These charts and others are irrelevant, Tufte implies. Worse, because no chart explicitly correlates cause and effect, the data just hangs there, leaving us wondering about the cause of such damage, but not knowing what to think. For all they tell a viewer, the damage could be caused by anything, a design flaw, God's will, what have you.
Second, no charts explicitly relate compromise of the O-rings to temperature, but the charts that implicitly correlate the two variables are misleading. 'Displays of evidence, as Tufte claims, implicitly but powerfully define the scope of the relevant, as presented data are selected from a larger pool of material (Tufte, 43). The chart entitled Blow By History defines the scope of what is relevant by focusing on 'blow-by (not erosion) and temperature for two launches, STS 15 [on January 24, 1985] and STS 22 [on October 30, 1985] (Tufte, 43). Focusing on blow-by invited the rhetorically devastating...comparison of STS 15 and STS 22 (Tufte, 42). The former flight at 53°F [STS 15] barely survived with significant erosion of the primary and secondary O-rings on both rockets as well as blow-by' while the 75°F launch [STS 22] had no erosion and only blow-by (Tufte, 42).
Had the engineers focused on the more common erosion, Tufte is arguing, STS 22 would not have been a counter-example to their argument (Tufte, 42), but in fact they set themselves up with a weak and misleading argument from analogy: STS 15 was launched when the O-rings were 53°F. There was very significant blow-by in STS 15. Therefore, no flights below 53°F should be permitted.
An argument relating what happens in a single instance to other instances is inherently weak. It is even weaker when the instance itself is problematic. It is a measure of how weak such an is by its very nature--that a single counter-example is as weighty as final evidence. So any flight above 53°F with compromise to the O-rings serves to undermine the implicit assumption of the conclusion, namely, that the rate and extent of compromise to O-rings rose as temperature declined (Allison,3). It is for that reason that STS 22 becomes a devastating counter-example, given its launch at 75°F and the blow-by that occurred. By Tufte's understanding of what the engineers were thinking, their argument should read like this if they put in all the data that focussing on blow-by made relevant?
STS 15 was launched when the O-rings were 53°F. There was very significant blow-by in STS15. STS 22 was launched with the O-rings were 75°F. There was significant blow-by in STS 22. Therefore, no flights below 53°F should be permitted. No wonder Tufte says that the engineers didn't quite know what they were doing, and they were doing a lot of it. Displayed in this way, the argument attributed to the engineers looks (and is) pitiful indeed, and as one reads through Tufte's account, one cannot help but wonder how the engineers could have convinced themselves, let alone anyone else.
Their first mistake, Tufte is claiming, was to misidentify the effect to which temperature ought to be related. The effect is not blow-by, but erosion, he claims. If they had gotten the effect right, he is arguing, at least their weak argument would not have been subject to such a devastating counter-example. For STS 22 had blow-by, but no erosion.
But that mistake was compounded by another, at least equally fatal error, Tufte claims. What is conspicuously missing from the charts the engineers presented and thus missing from the argument the engineers mounted is any attempt to correlate what their recommendation implies are causally related, namely, damage and temperature. Missing, first, are 92% of the temperature data, for 5 of the launches with erosion and 17 launches without erosion (Tufte, 43). Second, as the second conjunct implies, missing as well was any information about the launches without damage. We cannot begin to verify a claimed causal relationship without considering what is true of the supposed cause when the claimed effect is missing. As Tufte rightly puts it, The flights without damage provide the statistical leverage necessary to understand the effects of temperature (Tufte, 44). Third, and worst, only seven charts contained information about temperature and O-ring anomaly, but no single chart contained data on both in relation to each other(Tufte, 45; quoted from Lighthall, 65).
Had the engineers been thinking clearly, Tufte claims, they would have attempted to show the relation for temperature and compromise on all flights, and that attempt would have cued them into the need for presenting both all the temperatures and the different effects on the O-rings. They really needed only one chart, the scatterplot. But their failure to think through that they were arguing that the colder it gets, the more likely O-ring compromise, led them into a failure of presentation that had momentous consequences (Tufte, 45).
Tufte's claim is that the engineers were guilty of flawed reasoning in two ways:
- They misidentified the effect they were trying to prevent.
They were not thinking clearly enough, Tufte claims, even to identify that it was erosion, not blow-by, that should have been the focus of their concern.
- Having misidentified the effect, they proceeded to a generalization (do not fly below 53°F) from one example where both blow-by and erosion occurred.
They thus opened themselves up to STS 22's being a devastating counter-example. How could they have recommended not flying below 53°F on the basis of one instance when the same problem they claimed they were trying to prevent-- blow-by--occurred at 75°F? Poor reasoning, indeed!
Their failure to provide a scatterplot resulted in the Challenger's launch. So the engineers, thinking unclearly and representationally incompetent, are ethically responsible for the Challenger's failure and the death of the astronauts. Or so Tufte tells it.
What the Engineers were Trying to Prove
Were the engineers so confused that they misidentified the effect and so invited the rhetorically devastating--for those opposed to the launch--comparison of STS 15 and STS 22? Tufte claims that had the engineers not directed even their own attention away from the more common erosion to blow-by, STS 22 would not have presented a weakness for their argument (Tufte, 42). As one engineer said that evening, We had blow-by on the hottest motor [rocket] and on the coldest motor (quoted by Tufte, 42). Yes, the other engineers should have responded, but only on STS 15 did we have extensive erosion. They did not because their own charts confirmed and encouraged their mistaken concentration on blow-by--or so Tufte would have us believe.
Tufte rightly says that representation defines the database (Tufte, 43). It determines what is relevant and irrelevant to making a decision. Tufte refers throughout to O-ring distress and O-ring damage as the crucially relevant feature and begins his analysis by stressing the failure to assess the link between cool temperature and O-ring damage on earlier flights (Tufte,40). His way of representing the object of concern is as much definitive of the database he thinks relevant as the engineers. He thus ignores blow-by and concentrates on O-ring stress, or damage--a summarization, as he puts it, of the various ways in which the O-rings were themselves harmed (Tufte, 41).
An examination of STS 22 (on October 30, 1985) will help us understand that Tufte is making a mistake here in concentrating upon O-ring damage. When we look at the scatterplot chart Tufte thinks the engineers should have provided, we find that the index is marked O-ring damage and that STS 22 is given a 4 in the order of magnitude, a score, if we may call it that, which summarizes the damage to the primary and secondary O-rings in the nozzle joint. But this score fails to reflect the kind of problem STS 22 presented.
To understand its significance, we need to compare it with STS 15. The latter was a red flag because that was the first time we had actually penetrated a primary O-ring on a field joint with hot gas, and we had a witness to that event because the grease between the O-rings was blackened just like coal (Boisjoly in Vaughan, 155). In STS 15, the primary O-ring was penetrated completely, and the secondary O-ring was impinged, though not eroded, with the hot gases leaving a residue of burnt grease on it. The indication of blackened grease on STS 15 from hot combustion gas blow-by was 80 ° arc length on one case joint and 110 ° arc length on another case joint. By comparison, on STS 22, the blow-by indication was not a homogeneous black, but a light gray color with a much smaller arch length of 30 to 40 °. STS 22 was significant for two reasons.
First, the differences in the amount and color of the grease between STS 15 and STS 22 resulted from differences in the magnitude of the blow-by. The darker the color, the greater the amount of blow-by. Since STS22 was launched with an O-ring temperature of 75°F and had experienced a small amount of blow-by and STS 15 was launched with an O-ring temperature of 53°F and experienced a massive amount of blow-by- the conclusion to draw was that the lower the temperature, the greater the amount of hot gas blow-by and the closer the booster joint gets to complete failure.
Second, the cause of the hot gas blow-by in STS 22 was that the O-ring failed to seal momentarily due, it was reasoned, to the faulty joint design. Tufte's chart is thus misleading in that it fails to take into account the real magnitude of the O-ring and joint damage characteristics. If we consider only the erosion of the primary and secondary O-rings, as Tufte's phrase "O-ring damage" suggests we should, the damage index might be a 4. But the crucial feature of STS22 is that the primary O-ring did not seal, subjecting the secondary O-ring to erosion. The secondary O-ring was originally meant to be a redundant ring, there only as a safety precaution. Now with hot combustion gas blowing by the primary 0-ring, the secondary O-ring is forced to act as a primary O-ring, and if it did not seal or were eroded through, the results would be catastrophic. Tufte's chart fails to score the changed status of the secondary O-ring. But the engineers did take note of it and were very concerned. Redundancy was lost.
By concentrating on O-ring damage, Tufte completely misses the object of the engineer's concern, namely, that the O-rings might not seal at all, allowing hot gases to burn through the side of the rocket booster.
How could Tufte have be so confused? One reason is that he apparently thinks blow-by is soot, or so his pairing of the two--soot (blow-by)--would lead even a careful reader to assume (Tufte, 42). But soot is not in itself damaging either to the O-rings or to the success of a flight. So he thinks the engineers focused not only on the wrong effect, but on an effect that, he must think, has no impact on the safety of a flight.
But Tufte has mistaken an effect of blow-by for blow-by. Blow-by occurs when hot gases blow by an O-ring which has failed to seal fully in time. When an O-ring does not seal fully, a gap exists through which the hot gases of the rocket can pass, burning off the grease on the O-ring and impinging on the secondary O-ring, depositing there what is left of the combustion and burned grease, namely, soot. The soot is a causal effect of the hot gases blowing by an O-ring and heating up the grease that coats them. So blow-by is not soot, and as the engineers knew, it is potentially catastrophic.
Which would be more damaging to a flight--an O-ring being eroded or an O-Ring not sealing? These are not mutually exclusive problems, of course. If an O-ring does not seal, it is subject to both impingement erosion and by pass erosion, and the O-ring material gets removed...much, much faster (Boisjoly, quoted in Vaughan, 155). And if an O-ring is eroded through, then it does not matter whether it was sealed or not. But if an O-ring is eroded, and not eroded through, then it was sealed--and held. That it was only eroded is evidence that it did seal and held. But if an O-ring does not seal at all? That could be catastrophic--as catastrophic as an O-ring sealing and then being eroded through. And if both the primary and secondary O-rings failed to seal? That would be catastrophic.
The soot on the secondary O-ring in STS 15 occurred because the hot gases blew by the primary O-ring--which did not seal in time. Boisjoly suspected that the cold temperature was causally implicated in its not sealing in time, the O-rings having lost resiliency because cold, and so he and the other engineers were concerned that with even colder temperatures, neither O-ring would seal and the hot gases would blow by both O-rings, burning through the casing of the rocket booster and causing a catastrophic failure. They believed that the O-rings could withstand even severe erosion for the brief time they would be subject to erosion if they did in fact seal. So the worry was not that an O-ring would seal and then be burned through. The worry was that the O-rings would become so inflexible in the cold that they would not seal, and then it would not be their erosion that would matter, but the hot ignition gases blowing past them and compromising the casing itself and thus the flight.
Tufte is correct in saying that STS 22 experienced not much erosion, only blow-by, but blow-by was the main object of the engineers' concern. If blow-by occurred at 75°F, then the primary O-rings could not be depended upon to seal at 75°F, and if a primary O-ring was not sealing at 75°F, then the shuttles were at risk of catastrophic failure at what anyone would consider a normal temperature. No wonder the engineers were concerned about STS 22 and took it to be an essential part of their database.
The engineers were not thinking the way Tufte thinks they ought to have been thinking, and that was a good thing. They would have misrepresented the problem they faced had they used Tufte's scatterplot. The vertical axis on Tufte's scatterplot is mapping the wrong data, making a database of the wrong effect and directing attention away from what is relevant to making a decision.
But misidentifying the effect is not the only mistake Tufte has made the horizontal axis on his scatterplot is also wrong. But to understand fully how Tufte has gone wrong and to come to understand why the engineers made the recommendation they did so that we can properly assess their reasoning, we need to understand both the test and field databases the engineers had about 'O-ring damage' and the background information they had about what they did after the launch of STS 15 the previous January.
The Engineers' Test Database
The fixed tests--Tufte points out that the chart entitled History of O-ring Temperatures contains four test motors that never left the ground and so are not to the point (Tufte, 43). They were, he says,
All fixed rockets ignited on horizontal test stands at Thiokol, never undergoing the stress of a real flight. Thus this evidence, though perhaps better than nothing (that's all it's better than), is not directly relevant to evaluating the dangers of a cold-weather launch (Tufte, 43).
It seems a common mistake to think that the tests of the fixed motors were not relevant to evaluating the effectiveness of the O-rings sealing under the stresses of a real launch--whether in cold temperature or in warm. If one has that thought, one must wonder why the tests were conducted. In fact, however, the tests subjected the motors to more stresses than they would ever experience in flight.
The booster segments are not rigid, but highly flexible, settling out of round under their own weight, for instance, when transported on their sides. Vertical and stacked, in position for flight, the greatest stress comes from the hot gases against the inside of the booster rocket and occurs only in the first few seconds when the rocket is lifting off the launch pad. Strapped down on their sides and fired, the rocket bounces, subjecting the joints to additional stresses continuously as long as the rocket fires. Fixed and fired, on its side, the rocket will tend to become out of round, and its elliptical shape affects the gap created between the joints, rendering blow-by and erosion more likely. In addition, during actual launch, the booster bending loads occur for seconds while a fixed firing subjects the rockets to bending loads for the entire two-minute burn. For these reasons, it was safe to conclude that if a ground firing test was successful, the boosters were qualified for flight.
It is an additional piece of evidence that the four tests were held at temperatures between 47°F and 50°F. If the fixed rockets were subject in tests to far more stress than the rockets would get at launch, at colder temperatures than any launch to date, and the O-rings held, as they did, the tests seem far better than nothing for assessing whether the O-rings work effectively. They provide evidence that even at temperatures lower than 53°F, the O-rings hold.
The plate experiment--After STS 15, at the end of February and beginning of March 1985, Arnie Thompson performed a simple experiment to test O-ring resiliency indifferent temperatures. An O-ring was placed in a flight size groove in a flat plate and compressed...0.040 inches (1.02mm) with another flat plate. After temperature conditioning of the assembly, the plates were separated 0.030 inches (0.76mm) at a 2.0 inch per minute rate to simulate a flight rate of approximately 3.2 inches (8.13cm) per minute (slightly unconservative). (Boisjoly, 1)
What did the test show? There was no loss of contact at 100°F, but a loss of seal contact for 2.4 seconds at 75°F and in excess of 10 minutes at 50°F (Boisjoly, 1). These tests showed that the O-rings were not capable of filling the gap between the tang and the clevis created at launch insufficient time even at 75°F. 2.4 seconds was more than enough time for combustion gases to blow by the O-rings as they attempted to seal.
Thus, long before Challenger, the engineers knew both that the O-rings were not capable of sealing properly even at what no one would consider a cold temperature and that cold aggravated an already catastrophic problem.
A mixed bag--the military specification for Viton stated that it could be used at a temperature as low as -50°F, with the caution that verification is required in a specific application. But Thompson's experimental verification showed that Viton is not resilient enough even at 75°F to prevent disaster. Yet the fixed tests at temperatures below 53°F were successful, with the seals subject to far more stress, and stresses of different sorts, for a much longer time, than they would be in a launch. Perhaps the pressure of the hot gases against the sides of the booster rocket worked to seal the O-rings. In any event, whatever the cause of the successes and failures, the test data regarding resiliency of the O-rings presented the engineers with a mixed bag, determining no definitive conclusion by itself about the use of Viton in O-rings during an actual burn.
But we need to supplement the test data with the engineers field data if we are to have a more accurate picture of the epistemological position the engineers were in.
The Engineers' Field Database
We need to sort out what the engineers knew independently of the test data they had. One set of field data came through the seven instances of blow-by and/or erosion on the shuttles before the Challenger. It is important to appreciate the difference between seeing now all the data about the set of flights prior to the Challenger launch and seeing the data about each launch as they occurred. The engineers were in the midst of an unfolding process, and as they responded to problems with the shuttles, what stood out to them, and would stand out to any one engaged as they were, may well differ from what would stand out to us now with all the data at hand.
An Historical Narrative
The first problem involving the O-rings occurred in the second launch, STS 2. There was erosion of 0.053" of the primary O-ring in the right SRB's aft field joint (Vaughan, 121). Blow holes had formed in the putty when air trapped in the joint was compressed during joint assembly, and hot gases blew through the weak spots. But that occurred in only one of the 16 O-rings on the two boosters, and the conclusion was that the erosion was caused by a deficiency in the putty in only that location, unrelated to the O-rings (Vaughan, 121).
Nine successful launches followed, and then, on August 30, 1984, blow-by occurred in the nozzle joint of STS 12, with erosion on two primary O-rings and soot behind a primary ring. The soot behind the first O-ring was an indication that hot gases had penetrated behind that ring and put the secondary O-ring at risk, but that there was only a small amount of soot proved that the period during which hot gases passed the primary was short, verifying calculations that penetration by hot gases was a self-limiting phenomenon (Vaughan, 143).
The O-rings are tested before flight to determine whether they are properly sealed. The test requires putting air under pressure between the primary and secondary O-rings. That pressure ensures that the secondary O-ring is in place because it pushes that O-ring against the outer walls of its retaining groove, but if the pressure were high enough, it could push the primary O-ring away from its retaining groove sealing position. The tests up to and including STS 12 were made at 50 psi, but Leon Ray of NASA asked himself whether the putty might hold at that pressure. If so, the pressure would not be testing whether the O-rings were incapable of sealing because contaminated in some way. The air might get past a primary O-ring, proving that it was not properly sealed, but be held back by the putty so that those doing the tests would not know that the O-ring was not sealed.
A series of pressure tests down to 40°F indicated a problem with using only 50 psi, and so the leak check pressure was changed to 200 psi to ensure that the putty did not mask an O-rings not being sealed. Two successful launches followed before blow-by reached the secondary O-ring in STS 15 in the 100-year-cold in January 1985.
But the engineers had anticipated that a 200 psi check would push the primary O-ring out of its groove and so increase the likelihood of blow-by and erosion. Because they also thought that any blow-by of the primary O-ring was self-limiting, they judged this a tolerable risk and so took no corrective action after STS 15. But then came a flight in April (STS 17) which saw the...most extensive blow-by on a primary O-ring to date (Vaughan, 162). Erosion was 0.068 and so was outside the experience base of STS 2. But it was on a nozzle joint, and that design was different from the field joint design because it had a very safe secondary O-ring. It was a face seal between two metal surfaces clamped together with 100 1 1/8" diameter bolts. Yet the blow-by should not have occurred.
The testing pressure was decreased to 100 psi. Tests had been done showing that the putty could withstand up to 150 psi so that any test at that pressure or lower could mask the failure of an O-ring to seat. The engineers at Morton Thiokol and NASA recommended 200 psi. But NASA managers with the support of Morton Thiokol managers selected 100 psi as the leak test value.
Then came STS 22. At 75°F, the nozzle joint primary O-ring burned completely through with erosion of 0.171, exceeding both the experience base and the safety margin (Vaughan, 163). Because .09 is the maximum erosion that can occur if the primary O-ring seals, the judgment was that the nozzle joint's primary O-ring had never been in proper position to seal(Vaughan, 164). Some quality flaw in the installation--a hair or a piece of lint could do it--had occurred, and the 100 psi nozzle leak check had not detected that the ring was not in proper sealing position (Vaughan, 165). The pressure check was returned to 200-psi and remained there for all subsequent flights, including Challenger.
Arnie Thompson suggested thicker shim sand larger-diameter O-rings, but only the shims were added (yet see Vaughan, 179). There followed four successful launches before troubles again surfaced.
The launch on October 30, 1985 found soot behind two primary O-rings. Then, after one more success, the launch of Colombia on January 12, 1986 produced erosion at three joints. But that erosion was within the experience base and not unexpected given the increase in the pressure check to 200 psi (Vaughan, 285).
A summary of the history
Seven troublesome launches occurred before Challenger -- STS 2 (11.12.81), 12 (8.30.84), 15(1.24.85), 16 (4.12.85), 17 (4.29.85), 22 (10.30.85) and 24 (2.12.86). STS2 and STS 17 had causes unrelated to the composition of the O-rings. In the five other cases, what the history of incidents suggests is what the engineers in fact did. Each time a joint exhibited a problem found at disassembly after a flight, the problem was studied and assessed in preparation for the next flight. They looked each time for a cause for an effect, and they were successful with their fix. The problem either disappeared (as it did after STS 2) or a new problem appeared which was not unexpected given the fix.
At only one point in the history is temperature ever considered a possible issue. Until STS 15, none of the damage exceeded the 0.053" found after STS 2, and so flights were occurring within the field database created by STS 2. That more hot gases blew by the primary O-ring in STS 15 was a surprise, and Boisjoly suspected that the subsequent erosion was outside the parameter set by STS 2 because the cold weather affected the resiliency of the O-ring.
Lessons from the history
The troublesome effects the engineers saw in the history of shuttle flights seemed random--in two different ways. First, different joints were involved. Sometimes the problem occurred in a forward joint, sometimes in a center joint, sometimes in an aft joint, and sometimes in the nozzle joint. Second, different positions on each joint were involved. No one location of the joint cross section was singled out by the troublesome flights.
The most likely cause of the problems, if there were a common cause, would seemingly have to be something that could vary as the problems varied. A suspect whose potential for failures could match the randomness of the effects was the putty, with its variable behavior. If the putty failed at any one point, all the internal pressure would be concentrated at that one point rather than being evenly distributed around the inside perimeter of the rocket. Indeed, at one time it was suggested that the putty be removed to ensure the equalization of the pressure from the burn.
Putty formulations had changed during the flights due to the EPA's banning asbestos from the original putty. Replacements were found, but it was clear that all of them bordered on being unusable in a normal ground environment. For instance, putty in the high humidity at Cape Kennedy needed to be placed in freezers and removed only just prior to use because otherwise it would become too soft and sticky to put in place. When used in Utah, however, with its low humidity, no such precautions were necessary. In any event, it was unclear, for instance, whether the putty varied from batch to batch, whether the lay up from flight to flight varied, or whether the temperature or humidity affected the putty on a flight.
The engineers requested testing of the putty, but none was ever approved. The lesson is that the engineers and the rest of us are ignorant as to whether the blow-by and erosion were the result of the increase to 200 psi or whether the putty was the culprit or whether a combination of the two was the a use or whether some other factor was the crucial variable.
The Engineers' Epistemic Position
We have been examining the epistemic position of the engineers to determine what they knew or believed, or should and could have known or believed, about the shuttle problems. We need to add to the mix of problems that another striking feature of the history is the crescendo of problems that suddenly surfaced. From the first flight in 1981 until the end of 1984, two flights had difficulties, and as the subsequent history of successful flights indicated, both times the problem was explained and resolved. Then came the 100-year cold of January 1985, and within a year, there were four more troublesome flights. So the engineers found themselves in the following position in the summer of 1985: They knew that there were potentially catastrophic. They did not know the cause of the problems.
Given this, they did what they were professionally and ethically obligated to do:
- They informed those in authority--After the problems with STS 15 in January 1985 and the two flights in April, the engineers were rightly concerned, and on July 31, 1985, Roger Boisjoly sent a memo to the Vice President of Engineering at Morton Thiokol pointing out that if the blow-by problem of STS 17 were repeated in a field joint, [the result would be a catastrophe of the highest order--loss of human life (Boisjoly, 4). And during the July/August time period, NASA headquarters asked MortonThiokol
to prepare and present a summary of problems with all the booster seals on August 19, 1985. This was done....(Boisjoly, 4).
NASA's judgment was that despite the problems, flights would continue while a redesign was in progress. The problems were judged not so severe as to require the two-year delay in flights that would occur were they to wait for a new design to be ready.
NASA was thus aware of the difficulties with the shuttle design, and the engineers knew that NASA and all the other interested parties, including the managers at Morton Thiokol, knew there were problems. So when the engineers gathered together their charts to make their recommendation the night before the Challenger launch, they went into the room to remind everyone in the chain of command what everyone already knew. The charts were not new to anyone, and the information in them and the implications of that information were not news.
It is important to note another implication for when we consider the engineers' reasoning for their recommendation the night before the Challenger launch. NASA's decision in August to continue the flights, despite its now knowing there were potentially catastrophic problems and no known cause, made futile the engineers later recommending that no further shuttle launches should occur at any temperature.
- They tried to determine the cause--ignorant of the cause, and trying not to overlook any possibility, Roger Boisjoly compiled a list of data in September 1985 that the engineers thought they needed to try to get a fix on what variables were relevant to the effects they were observing. They were ignorant of a great deal that had not previously been thought relevant. In particular, they did not know any more about temperatures at launch than Tufte remarks on--that the O-rings on STS 15 were calculated to be 53°F and that the ambient air temperature for 61-A was 75°F (Tufte, 44). These two pieces of data were in the charts the engineers presented at the teleconference.
Tufte never says that the engineers had the temperature data at hand, but his saying that they should have presented the scatterplot he give implies that they could have presented it, that is, that they had that data. And, as we have seen, that is a natural way to read him. In describing his work, one writer says that Tufte goes through the charts [with heartbreaking thoroughness and demonstrates how one simple graph of the data they had at hand--information about the failure of the booster rockets O-rings at various temperatures--would have alerted them to the dangers they faced (Martin, our italics, 276). But, in fact, to repeat, they did not have that data--though not for want of trying.
Temperature was not thought to be a relevant variable, and it certainly did not seem to be a relevant variable given the norm in Florida. Boisjoly's suspicions regarding the effect of cold on the O-rings in the flight in January 1985 changed its status so that it became a relevant variable. But it had not been collected as a matter of course and so was not readily available. In addition, finding out the ambient air temperature at time of launch is not the same as determining the temperature of the O-rings at that time.
In the experiment where an O-ring was placed in a groove on one steel plate and compressed by another, there had to be temperature conditioning of the assembly (Boisjoly, 1). That is, the engineers had to be sure that all the components were at the chosen temperature for the test. Were an O-ring taken from storage and put immediately to a test at 100°F, we would not obtain accurate information about the resiliency of the O-rings at 100°F. Just so, if we have the ambient air temperature at the time of launch, we shall still need to calculate the temperature of the O-ring. That is what the engineers had to do for STS 15. The shuttle had been sitting out in temperatures below 50°F for some days, and the calculation was that the O-ring was 53°F when the ambient air temperature at launch was 67°F. The O-ring temperature of STS 22 was later calculated to be 75°F when the ambient air temperature was 78°F (Boisjoly, Figure 8, 6). So even if the engineers had the data about ambient air temperatures, they would have needed more data to calculate with an acceptable degree of probability the temperature of the O-rings. How long was the shuttle on its pad? What were the variations in temperature during that time? How great was the variation? How long was the temperature at this degree, how long at that? And so on. Calculating the O-ring temperature for each flight would have been demanding of time and energy--and not a worthwhile expenditure of a valuable resource, time, when the variable was not thought relevant.
The data necessary for a calculation of O-ring temperatures was thus not collected all along during the shuttle history. And when Boisjoly asked for that data in September, along with much other data, any one of which might have been the crucial missing piece to explain the anomalous cause, it was not supplied. In fact, the engineers received none of the data they requested.
So, to summarize, the engineers did what they were professionally and ethically obligated to do. (1) They informed those in authority, and (2) they tried to determine the cause. Arnie Thompson's steel plate experiment was part of the effort to determine the cause, and his suggestions to add shims and increase the diameter of the O-rings were part what they did to make the best they could of a bad situation. (3) So they did what they could to mitigate the problem given NASA's decision to continue the flights despite their knowing of the danger of catastrophic failure.
The Engineers' Reasoning
It is always with trepidation that one should try to reconstruct how it is that a decision was made, particularly when the decision is a joint decision of different individuals who may have had different understandings and intentions, when the decision was conveyed under hectic conditions and when those making the decision were not called upon to justify it until long after it was made. But the engineers' epistemic position at the teleconference gives us a clue to their reasoning.
First, the blow-by on STS 22 was a crucial piece of field confirmation of Arnie Thompson's plate experiment, and the differences in the amount and color of the soot in STS 22 and STS 15 was evidence that the colder it got the less resilient the O-rings, another piece of field confirmation of the plate experiment. It does not take a rocket scientist to fear a line of increasing blow-by from75°F to 53°F to 29°F and thus an increasing risk of catastrophic failure. The argument here is not an argument from analogy, using a single problematic case as its basis, but an inductive inference based on a correlation between increasing blow-by at lower temperatures and a theory about what was wrong.
This argument is not in and of itself very strong. Two instances of a correlation do not generally provide powerful grounds for an inference. On formal grounds, that is, no one ought to accept the conclusion that blow-by will increase at 29°F. But in conditions of uncertainty and risk engineers operate with a decision-procedure that the rational choice is to avoid unusual risk. Using that decision-procedure, the argument is far more powerful. There seems to be increasing blow-by as temperatures drop, something witnessed in the flights of STS22 and 15 at 75°F and 53°F, and that increased blow-by is consistent with what was discovered in the plate experiments. Both experience and experiments suggest that if we are to be risk-averse, then we ought not to recommend launching a shuttle at a colder temperature, particularly at a temperature so much colder than 53°F as the 29°F projected for Challenger at launch.
But, second, the engineers knew that they did not know that decreased temperature was correlated with greater blow-by. They could at most infer the likelihood of an increased risk. But they were arguing with full knowledge that the design was flawed and with known ignorance--known to NASA and the Morton Thiokol managers as well as to them--of the complete causes of the blow-by. Without clarity about the causes of the blow-by and subsequent compromise to the O-rings and the flight, but knowing that at 53°F they had more significant damage than at 75°F, they saw what should seem to be obvious to anyone that evening. There is a huge jump between a flight at 53° and one at 29° and so an increased unusual risk (see Chart, Tufte, 43)
Third, this ignorance of the cause of the problem play another role in the engineers' reasoning. It is always at a risk that we attribute a single view to a group of individuals, and it is even more risky when the view is never fully articulated and put to paper. But hovering in the background during the teleconference seems to be the engineers belief that no shuttles should be launched until the problem was found and fixed. If blow-by occurred at 75°F, then it could seemingly occur at any temperature, and the secondary O-ring becomes primary. That is unacceptable. But the engineers had made this argument to NASA in August and lost. So they were precluded--because it was useless--to make it again now.
So they recommended that there be no launch outside their field database. As Tufte puts it, in a line which sums up the general premise from which the engineers were arguing, though Tufte does not recognize that.
This launch was completely outside the engineering database accumulated in 24 previous flights (Tufte, 45).
Engineers distinguish carefully between test data and field data--experimental evidence and experiential evidence. They are cognizant, as Tufte rightly implies they should be, that what is shown interests may not hold under real conditions. So though they knew that the tests in Utah showed that the O-rings had held without blow-by or erosion under cold down to 48°F, they also knew that these were experiments. What their experience showed was that at 53°F they had significant blow-by--enough to cause massive damage to the primary O-ring and impinge on the secondary O-ring.
One premise of their decision-procedure is that experience trumps experiments and the only experience they had of sending off a shuttle at a low temperature--for STS 15, where the ambient air was 66°F, but the O-ring temperature was calculated to be at 53°F --resulted in blow-by reaching the secondary O-ring. They had done no experiments of what would happen when the temperature was in the high twenties or low thirties, and so the question arises. What ought one to do when there are no experiments or experience one way or the other regarding a particular instance of a phenomenon, cold, that may be relevant to flight safety, but when we do have experience that some degree of the phenomenon is a source of incidents?
The answer ought to be as obvious as Tufte thinks the answer from his scatterplot to be. A launch at the expected temperature of 29°F is so far outside the field database that anyone with sense, averse to risk, would not launch the Challenger. And, indeed, the managers at Morton Thiokol were convinced until NASA asked for proof that Challenger was not flight-ready. The engineers' job was to make a recommendation about whether it was safe to launch, not to prove that it was not safe to launch. By shifting the burden of proof, NASA was shifting from a decision procedure that was risk averse to one congenial to high fliers, willing to take a risk, even if the results might be catastrophic, unless it could be proven that what created the risk would in fact occur.
One might still wonder why the engineers did not correlate temperature and blow-by when their very recommendation--not to fly below 53°F --tied together risk and temperature. But even if they had the relevant data, they would not have tried to construct a scatterplot for temperature and incidents because it would never have occurred to them that they needed one or that one would be helpful. They would have had a scatterplot' with four pieces of data--the differing amounts of blow-by at 75°F and 53°F. One does not need a scatterplot to make the point that it is risky to fly at 29°F given what had happened at 75°F and 53°F, and, in any event, given that they were not sure that they knew the cause of the blow-by problem, their basic premise was that Challenger would be flying beyond their database. That evening, regarding the Challenger launch, the relevant feature that was outside their database was temperature, and so, quite reasonably, their recommendation reflected the problems of flying at a temperature outside their database--particularly one so much colder than any previous flight.
Tufte's Misrepresentation
Tufte's concern is with the visual representation of data, but, obviously, we can also represent through a narrative. Historians do it. Journalists do it. We all do it--including Tufte. And just as there are criteria for graphic representations, criteria that Tufte nicely lays out in his works, there are criteria for narrative representations, criteria that can vary depending upon what it is we are representing.
In representing historical events in which the actions (and omissions) of historical personages are the focal point, for instance, we try to take on their point of view--their place in time and in space--as best we can. It would be an odd kind of history indeed which faulted Caesar for not foreseeing his death at the hands of Brutus or queried why, given what was going to happen, Robert E. Lee ordered Pickett's charge at Gettysburg. Such criticisms come from taking our point of view, assuming that those historical personages were somehow privy to our understanding of the results of their acts. The minimal condition required of us in writing of historical personages is that we restrict our database to what was, or ought to have been, available to those who were deciding what to do. We may still find fault with what they did. Robert E. Lee's order a Gettysburg seems misconceived even given what we know.
Tufte has said that he is not interested in history:
...I'm not particularly interested in who did what first, or development. Because it is one damned thing after another. It's unconceivable (Computer Literacy Bookshops Interview).
Tufte's judgment of what the engineers should have done the night before the launch requires an historical appreciation of where they found themselves. It was one damned thing after another and the frustrating part for the engineers is that they lacked the data--despite having asked and even pleaded for it--to back up their collective sense that the flight should not be launched at such a temperature. Tufte presumed wrongly that the engineers had full information. He presumed rightly that they acted voluntarily in making their presentation. So, given the conditions for judging ethically whether a person is morally responsible, he inferred from these two presumptions that they were incompetent. But Tufte has taken, as it were, a God's eye view of the data, faulting the engineers for providing only a few temperature data points and not connecting those up with the known effect properly. God is timeless. We are historical beings, and so we make decisions that reflect the data we have. We can do a good job of that or a bad job, and we can fail to have data we should have and could have, but it is ethically wrong to upbraid us for not making a decision not even God could have made if God were restricted to the only evidence we can obtain.
Those few data points were all the engineers had--despite their best efforts to get more. And they did not connect up those data points with temperature because they only suspected but did not know that cold and O-ring compromise were causally related and, as we have said, were not arguing that they were. So Tufte has thoroughly misrepresented the engineers' position. With the data available to them, and with NASA knowing as well as they that the design was flawed and that temperature might be a causal factor, they argued that they ought not to fly so far out of the field database. Tufte was right in ignoring the test data, but for the wrong reason. The engineers were trying to stick to the firmest evidence they had--the field database?
But data is not all that counts. As Tufte argues well, one can have the most powerful position possible for something and fail completely to convince anyone of it with a poor presentation. The presentation of the data and the arguments that inform it are crucial.
Tufte's scatterplot well represents the data he presents, but he has the wrong data. The scatterplot is preceded in his text by the following table.
The temperatures listed are marked Temperature in °F, with no indication of what they are temperatures of (Tufte, 44). Tufte does not indicate on this chart whether these are the ambient air temperatures or the temperatures of the O-rings , but in referring to a chart the engineers presented entitled, history of O-ring temperatures, which follows: he says,
While it was true that the blow-by on SRM 15 was on a cool day, the blow-by on SRM 22 was on a warm day at a temperature of 75° (temperature chart [referring to the above chart], second column from the right) (Tufte,42).
His assumption seems to be that the ambient air temperature and the temperature of the O-rings are the same--despite the engineers' chart indicating differences between the two. If Tufte is not making that mistake, it would be hard to explain either the scatterplot or his remarks on the table replicated above that precedes it. For both list temperature as one variable. The scatterplot refers to it as Temperature (°F) of field joints at time of launch, but the chart the engineers provided distinguishes between the ambient air temperature (the third column) and the temperature of the O-rings (the fourth column), giving known and calculated figures for STS 15 and 22 and predicted and projected figures for Challenger the next morning. If we compare the scatterplot with the chart, we can see that while two of the temperatures Tufte provides are of the O-rings at the time of launch, the other temperatures are of the ambient air at time of launch. Tufte has mixed apples and oranges--no way, as he himself would emphatically agree, to represent the data perspicuously.
So even if the engineers had the data in hand and had used a scatterplot, they would not have used the one Tufte provides. Tufte's has both coordinates wrong. The vertical axis should be blow-by, not O-ring damage and the horizontal axis should be O-ring temperature, not a mixture of O-ring temperature and ambient air temperature. It is Tufte here who does not quite know what [he] is doing, and [is] doing a lot of it (paraphrase of Tufte, 45).
Moral Responsibility
Were the engineers morally responsible for the Challenger disaster? If they had been Gods, with all the data readily at hand, we could be held no more responsible than NASA. But even that is to concede too much.
They lacked the power to halt the flights, and they exercised the only powers they had and did so in a timely manner. They brought the problem to NASA's attention. If we are going to make a moral judgment that they were wrong, we need, first, to keep in mind that they knew there was a problem and that they informed those in authority.
Second, we need to keep in mind that someone who makes a judgment based on lack of information is prima facie not morally responsible if there was a good-faith effort to obtain that information. And there was.
Third, we need to keep in mind that someone who tries to rectify the situation that may be causing the problem is less responsible than someone who ignores the problem, and the engineers did what they could given the cards they were dealt. They tried to gather more information to get a definite fix on the problem and, for instance, added shims as Arnie Thompson suggested.
And, fourth, we need to remember that they succeeded in convincing their managers--if only because they had a collective sense that a launch should not occur and they were, after all, the best positioned in the world to make such a judgment. They failed only because NASA refused to accept their recommendation and the managers at Moron Thiokol used and overturned their recommendation. This is not to say that their presentation was not flawed or that even if conceptually correct, could not have been better done. It is to say that they should not bear the moral fault for a flight they had recommended against when they should have, and under normal circumstances would have, seen their recommendation upheld.
Does Tufte bear moral responsibility for falsely accusing the engineers of an overriding intellectual failure (Tufte, 52)? For falsely accusing them of a scandalous discrepancy between the intellectual tasks at hand and the images created to serve those tasks (Tufte, 45)? For falsely accusing them of failing to save the lives of the astronauts by producing a scatterplot so clear that no one would have dared to risk the Challenger in such cold weather (Tufte, 52).
It would, of course, be wrong for us to criticize Tufte had he tried to obtain the information about what the engineers knew but could not obtain it for reasons beyond his control. But, as we have noted, all the information we have cited was available to Tufte had he sought it. We are not attributing to him responsibility for information he could not have known.
Perspicuous representation is an ideal to strive for, but Tufte has dramatically failed to achieve it himself in critiquing the Morton-Thiokol engineers. His narrative and scatterplot do his own thesis a disservice. It is not competent, and is morally wrong, to design a criticism that so badly misrepresents the position of those one is critiquing and so badly fails to capture the problem they were facing. The harm is magnified by the popularity of Tufte's work, by its adoption by schools of business, by his giving seminars to various professional groups and corporations on representation, and, when he does so, holding the Challenger case up as a paradigmatic example of what can go wrong when not achieving what he argues is the ideal. Any moral judgment of Tufte should be modified accordingly.
References
- Allison, Nicholas H., "Design You Can Count On", review on Amazon's webpage.
- Boisjoly, Roger, Ethical Decisions- Morton Thiokol and the Space Shuttle Challenger Disaster, ASME Proceedings (December 13-18, 1987).
- Computer Literacy Bookshops at Interviews with Edward R. Tufte, 1994 and 1997.
- Duncan, Ray, Absolute Power on Dr. Dobb's Electronic Review of Computer Books www.ercb.com.
- Lighthall, Frederick, "Launching the Space Shuttle Challenger: Disciplinary Deficiencies in the Analysis of Engineering Data", IEEE Transactions on Engineering Management, 38 (February 1991).
- Martin, Michael H., "The Man Who Makes Sense of Numbers", Fortune (October 27, 1997), 273-276.
- Tufte, Edward R., Visual Explanations: Images and Quantities, Evidence and Narrative (Cheshire, Connecticut: Graphics Press, 1997).
- Vaughan, Diane, The Challenger Launch Decision (Chicago: The University of Chicago Press, 1996).
Notes
Citation
Anonymous. . Representation and Misrepresentation: Tufte and the Morton Thiokol Engineers on the Challenger. Online Ethics Center. DOI:. https://onlineethics.org/cases/representation-and-misrepresentation-tufte-and-morton-thiokol-engineers-challenger.
